Many types of server applications and system-level services operate according to an event-driven paradigm; in other words, they monitor for events from one or more event sources and then process the events using event handling logic. In recent years, there has been an increasing need to scale the performance of such applications/services in order to handle very large numbers of concurrent events. This need, along with the proliferation of multiprocessor computer systems, has led many developers to implement multithreading for parallelizing event processing across multiple threads/processors.
One common approach for implementing multithreading in an event-driven application involves reserving a single thread as a poll thread and a pool of threads as worker threads. The poll thread monitors for incoming events on all of the active connections between the application and event sources, where each active connection is represented by an I/O handle (e.g., a network socket, a file descriptor, etc.). The set of monitored I/O handles comprise the poll thread's poll list. The poll thread typically implements this monitoring functionality using an event de-multiplexer, such as the select( ) or poll( ) system call available in UNIX. When the poll thread detects an event via the event de-multiplexer, the poll thread reads the event from its corresponding I/O handle and schedules a task to be performed for the event in a task queue. One of the worker threads thereafter picks up the task from the task queue and processes the task using an appropriate event handler.
While the foregoing approach is functional, it also suffers from a number of inefficiencies and disadvantages. First, traditional event de-multiplexers (e.g., the UNIX select( ) and poll( ) system calls) generally do not scale well with the number of concurrent I/O handles being monitored. Thus, if the size of the poll thread's poll list grows to be very long, the poll thread can become a bottleneck for overall application throughput and performance. Second, multiple worker threads may require access to the same resource (e.g., file, database table, etc.) simultaneously. This may occur if, for example, back-to-back tasks in the task queue require invocation of the same event handler. To account for this, the application developer must write synchronization code in order to avoid resource deadlocks and race conditions between worker threads, which significantly complicates application design and testing. Third, threads tend to block on synchronization primitives. This means that, in order to avoid running out of free worker threads (which are synchronized as mentioned above), the worker thread pool size must be fairly large. This, in turn, can lead to reduced performance due to context switching, cache thrashing, and other problems that occur when a large number of threads are constantly loaded and unloaded on a smaller number of processors.